Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change environment variable getters to error recoverably #86183

Merged
merged 4 commits into from
Aug 2, 2021

Conversation

inquisitivecrystal
Copy link
Contributor

@inquisitivecrystal inquisitivecrystal commented Jun 9, 2021

This PR changes the standard library environment variable getter functions to error recoverably (i.e. not panic) when given an invalid value.

On some platforms, it is invalid for environment variable names to contain '\0' or '=', or for their values to contain '\0'. Currently, the standard library panics when manipulating environment variables with names or values that violate these invariants. However, this behavior doesn't make a lot of sense, at least in the case of getters. If the environment variable is missing, the standard library just returns an error value, rather than panicking. It doesn't make sense to treat the case where the variable is invalid any differently from that. See the internals thread for discussion. Thus, this PR changes the functions to error recoverably in this case as well.

If desired, I could change the functions that manipulate environment variables in other ways as well. I didn't do that here because it wasn't entirely clear what to change them to. Should they error silently or do something else? If someone tells me how to change them, I'm happy to implement the changes.

This fixes #86082, an ICE that arises from the current behavior. It also adds a regression test to make sure the ICE does not occur again in the future.

@rustbot label +T-libs
r? @joshtriplett

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @joshtriplett (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 9, 2021
@rustbot rustbot added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Jun 9, 2021
@joshtriplett
Copy link
Member

joshtriplett commented Jun 10, 2021

I can see the argument for failing more gracefully. On the other hand, this seems more like an internal assertion, and an entirely avoidable one: it'll only happen if you pass an invalid environment variable name. Silently turning that into a None as though the environment variable simply doesn't exist seems unfortunate, and could miss flagging an actual logic error in someone's code.

On the other other hand, it is in a sense accurate that no variable named varname=morevarname\0yetmorename exists. 🙂

@inquisitivecrystal
Copy link
Contributor Author

inquisitivecrystal commented Jun 10, 2021

On the other other hand, it is in a sense accurate that no variable named varname=morevarname\0yetmorename exists. 🙂

This would be my view. I wouldn't even say "in a sense". If you look for the variable in the environment, is it there? It wouldn't be. Thus, it's arguably somewhat irrelevant that it couldn't exist.

That said, if you would prefer to keep the library unchanged, that's an entirely fair call. How would you like to proceed on this?

Edit: I just noticed that you nominated it for the libs meeting. I look forward to hearing the result!

@ChrisDenton
Copy link
Member

ChrisDenton commented Jun 10, 2021

I can see the argument for informing the programmer that a constant key name will always be invalid (i.e. never be set). However a panic at runtime seems very heavy handed considering there's no good way for the application to recover (except for catch_unwind if not aborting on panic).

@inquisitivecrystal
Copy link
Contributor Author

I can see the argument for informing the programmer that a constant key name will always be invalid (i.e. never be set). However a panic at runtime seems very heavy handed considering there's no good way to recover (except for catch_unwind if not aborting on panic).

Also, what if the end user is the one who provides the environment variable name, like in #86082 (in that case, the end user of rustc)? Then it's not the programmer's fault, and we're making them check specifically for a weird special case that only happens on certain platforms. Worse, there are platforms where this would be valid, so then they have to implement platform specific logic to handle information we already have available.

I don't really think the status quo is tenable. However, one could consider introducing a new API is a better course.

@m-ou-se m-ou-se changed the title Change environment variable getters to error recoverably Change environment variable getters to error recoverably Jun 23, 2021
@rustbot rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Jun 23, 2021
@m-ou-se
Copy link
Member

m-ou-se commented Jun 24, 2021

We discussed this yesterday in the libs team meetings.

From an API perspective, we think it's perfectly reasonable to treat things like = or \0 as any other environment variable that doesn't exist, since we are sure that no such environment variable can exist.

For the implementation: Instead of having os_imp::getenv return an io::Error and ignoring it in _var_os, it'd be better to make getenv just return an Option and not a Result. Then it's clear at the implementation of getenv that errors are not going to be handled.

/// `'='` or the NUL character `'\0'`, or when the value contains the NUL
/// character.
/// May error if the `key` is empty, contains an ASCII equals sign `'='`, or contains the NUL character `'\0'`.
/// May error when the value contains the NUL character.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any platforms where this is possible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows nulls aren't allowed but it is technically possible for a key to begin with an = (e.g. =key=value). However this is essentially a backwards compatibility hack used only by cmd.exe so as to have hidden variables for internal use. It is not documented and is not meant to be set or used by other code. That said, nothing actually prevents a program from doing so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are also some targets where the env is literally a hashmap. Sgx, for example. I'm not sure in what sense it's an environment at that point, but perhaps I'm missing something.

library/std/src/env.rs Outdated Show resolved Hide resolved
@inquisitivecrystal
Copy link
Contributor Author

Is there anything I should do about the documentation? I was just rereading it, and it doesn't feel like "errors" or "may error" is the clearest way to put things. I don't really know how to fix improve on that though..

@m-ou-se m-ou-se assigned m-ou-se and unassigned joshtriplett Jun 30, 2021
@bors
Copy link
Contributor

bors commented Jul 3, 2021

☔ The latest upstream changes (presumably #79965) made this pull request unmergeable. Please resolve the merge conflicts.

Before this, `std`'s env var getter functions would panic on
receiving certain invalid inputs. This commit makes them
return a `None` or `Err` instead.
@inquisitivecrystal
Copy link
Contributor Author

inquisitivecrystal commented Jul 6, 2021

I fixed up the docs and rebased.

@m-ou-se As far as I know, this is ready to merge.

@camelid camelid added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 23, 2021
@m-ou-se
Copy link
Member

m-ou-se commented Aug 1, 2021

Thanks!

@bors r+

@bors
Copy link
Contributor

bors commented Aug 1, 2021

📌 Commit d9752c7 has been approved by m-ou-se

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 1, 2021
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 2, 2021
Rollup of 13 pull requests

Successful merges:

 - rust-lang#86183 (Change environment variable getters to error recoverably)
 - rust-lang#86439 (Remove `Ipv4Addr::is_ietf_protocol_assignment`)
 - rust-lang#86509 (Move `os_str_bytes` to `sys::unix`)
 - rust-lang#86593 (Partially stabilize `const_slice_first_last`)
 - rust-lang#86936 (Add documentation for `Ipv6MulticastScope`)
 - rust-lang#87282 (Ensure `./x.py dist` adheres to `build.tools`)
 - rust-lang#87468 (Update rustfmt)
 - rust-lang#87504 (Update mdbook.)
 - rust-lang#87608 (Remove unused field `Session.system_library_path`)
 - rust-lang#87629 (Consistent spelling of "adapter" in the standard library)
 - rust-lang#87633 (Update compiler_builtins to fix i128 shift/mul on thumbv6m)
 - rust-lang#87644 (Recommend `swap_remove` in `Vec::remove` docs)
 - rust-lang#87653 (mark a UB doctest as no_run)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors
Copy link
Contributor

bors commented Aug 2, 2021

⌛ Testing commit d9752c7 with merge effea9a...

@bors bors merged commit 016612d into rust-lang:master Aug 2, 2021
@rustbot rustbot added this to the 1.56.0 milestone Aug 2, 2021
@lopopolo
Copy link
Contributor

lopopolo commented Aug 15, 2021

With this change, how does a user of env::var_os distinguish between an env variable name not existing and the underlying platform returning EINVAL?

I'm interested in being able to distinguish this behavior since I'm implementing Ruby's ENV which raises on invalid names:

I think in my ideal world, there'd a fallible version of env::var_os that doesn't squash to None or panic. Similarly, a fallible version of env::set_var (which still panics with invalid var names after this PR) would help here too.

With the panicking behavior prior to this PR being merged, I had to pull the EINVAL checking into my own code above the call to std::env::var_os: https://github.com/artichoke/artichoke/blob/daf01d7fa995ab42e21444f3834c49f4563d7301/spinoso-env/src/env/system.rs#L97-L127.

Additionally, I think this PR should be tagged with relnotes.

@lopopolo
Copy link
Contributor

Is it not a breaking change to turn documented panics into something else?

I reported the undocumented panics here #63456 and they were added to the docs as part of the API contract here #63461.

@RalfJung
Copy link
Member

Cc @rust-lang/libs-api for the questions raised above -- though this was approved by a team member.

@lopopolo
Copy link
Contributor

lopopolo commented Aug 15, 2021

Python has a similar behavior to Ruby.

root@e8db2cec4f95:/# python3 -c "import os; os.environ['a'] = 'abc\0xyz'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/os.py", line 681, in __setitem__
    self.putenv(key, value)
ValueError: embedded null byte
root@e8db2cec4f95:/# python3 -c "import os; os.environ['a\0b'] = 'e'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/os.py", line 681, in __setitem__
    self.putenv(key, value)
ValueError: embedded null byte
root@e8db2cec4f95:/# python3 -c "import os; os.environ['a=b'] = 'e'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/os.py", line 681, in __setitem__
    self.putenv(key, value)
ValueError: illegal environment variable name
root@e8db2cec4f95:/# python3 -c "import os; print(os.getenv('a=b'))"
None
root@e8db2cec4f95:/# python3 -c "import os; print(os.getenv('a\0b'))"
None
root@e8db2cec4f95:/# python3 -c "import os; print(os.getenv('a=b'))"
None
root@e8db2cec4f95:/# python3 -c "import os; print(os.environ.get('a\0b'))"
None
root@e8db2cec4f95:/# python3 -c "import os; print(os.environ.get('a=b'))"
None

root@e8db2cec4f95:/# ruby -e 'puts ENV["x\0y"]'
Traceback (most recent call last):
	1: from -e:1:in `<main>'
-e:1:in `[]': bad environment variable name: contains null byte (ArgumentError)
root@e8db2cec4f95:/# ruby -e 'puts ENV["x=y"]'

root@e8db2cec4f95:/# ruby -e 'ENV["x\0y"] = "a"'
Traceback (most recent call last):
	1: from -e:1:in `<main>'
-e:1:in `[]=': bad environment variable name: contains null byte (ArgumentError)
root@e8db2cec4f95:/# ruby -e 'ENV["x=y"] = "a"'
Traceback (most recent call last):
	1: from -e:1:in `<main>'
-e:1:in `[]=': Invalid argument - setenv(x=y) (Errno::EINVAL)
root@e8db2cec4f95:/# ruby -e 'ENV["xy"] = "a\0z"'
Traceback (most recent call last):
	1: from -e:1:in `<main>'
-e:1:in `[]=': bad environment variable value: contains null byte (ArgumentError)

How would one implement this API with the existing APIs in std? I know it's not a goal for std to replicate the APIs of other programming languages, but I expect Rust as a systems language to expose the details and limitations of the target platform when I need it to.

For APIs in std that call into libc functions that set errno, I would hope to have the option to use a fallible version. For APIs in std that call into libc functions that have different preconditions on their inputs than encoded in OsStr, I would hope to have the option to use a fallible version.

The previous behavior with panics was not ideal, but it did allow callers to observe failures and recover from them with catch_unwind.

@inquisitivecrystal
Copy link
Contributor Author

Is it not a breaking change to turn documented panics into something else?

I'm definitely not an expert, but I would think that making something that didn't work start working would be allowed. Of course, if it broke a lot of people's use cases, it's somewhat irrelevant...

@lopopolo
Copy link
Contributor

lopopolo commented Aug 15, 2021

This change alters the semantics of the env::var_os API to mask a previously documented and observable panic due to a limitation of the underlying platform. This may be the right change to make for env::var_os, but without a replacement API, maybe env::try_var_os or env::var_os_for_key, less information is available to callers than before.

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Aug 18, 2021
Fix environment variable getter docs

`@RalfJung` pointed out a number of errors and suboptimal choices I made in my documentation for rust-lang#86183. This PR should (hopefully) fix the problems they've identified.
@m-ou-se m-ou-se added the relnotes Marks issues that should be documented in the release notes of the next release. label Aug 18, 2021
@joshtriplett
Copy link
Member

joshtriplett commented Aug 18, 2021

We reviewed this in today's @rust-lang/libs-api meeting.

We appreciate the specific use case of needing to provide more nuanced validation for the purposes of implementing another API. However, we feel like the changes previously made here will be what the majority of users will expect; in particular, it looks like from the implementation of Artichoke that you're actually using the same semantic for keys containing = (returning the same result as if they didn't exist). Given that, we generally felt like doing this kind of validation separately in advance (or afterward iff the result is None) seems reasonable.

That said, we'd also be sympathetic to improving the convenience of working with OsStr (cc @dtolnay). And if there's an additional semantic here that's OS-specific (rather than something reasonably easy to check for all platforms such as containing = or \0), and a try-ish method would actually provide meaningful value, we'd be willing to consider such a method.

@lopopolo
Copy link
Contributor

Thanks @joshtriplett and Rust Libs API team for taking a look. I do agree that the changes made here better serve most users and what I'm hoping to do is a bit niche.

I'd be super excited to see improvements for the ergonomics of OsStr. I think I've seen an idea for getting something like the pattern API implemented for OsStr on internals a couple of times. Being able to do contains checks (like for this env var name and value case) and starts_with checks (like for checking for - and -- flags and switches in parsing command line arguments) would go a long way to avoiding round tripping through &str or &[u8].

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Oct 22, 2021
Pkgsrc changes:
 * Remove one now-longer-applicable patch, adjust a few others
 * Bump bootstrap requirements to 1.55.0.

Upstream changes:

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on
  aarch64-unknown-freebsd.][rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`,
  rather than end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`] This has no effect at present on dependency
  version selection.  We encourage crates to specify their minimum
  supported Rust version, and we encourage CI systems that support
  Rust code to include a crate's specified minimum version in the
  text matrix for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825] Rust
  now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu
  artifacts.][rust#88069] This improves the performance of most Rust builds.

- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 20, 2021
Pkgsrc changes:
 * Bump bootstrap kit version to 1.55.0.
 * Adjust patches as needed, some no longer apply (so removed)
 * Update checksum adjustments.
 * Avoid rust-llvm on SunOS
 * Optionally build docs
 * Remove reference to closed/old PR#54621

Upstream changes:

Version 1.56.1 (2021-11-01)
===========================

- New lints to detect the presence of bidirectional-override Unicode
  codepoints in the compiled source code ([CVE-2021-42574])

[CVE-2021-42574]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-42574

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]:
  https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.]
  [rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`, rather than
  end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`]
  This has no effect at present on dependency version selection.
  We encourage crates to specify their minimum supported Rust
  version, and we encourage CI systems that support Rust code to
  include a crate's specified minimum version in the text matrix
  for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825]
  Rust now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts.]
  [rust#88069]
  This improves the performance of most Rust builds.
- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666

Version 1.55.0 (2021-09-09)
============================

Language
--------
- [You can now write open "from" range patterns (`X..`), which will start
  at `X` and will end at the maximum value of the integer.][83918]
- [You can now explicitly import the prelude of different editions
  through `std::prelude` (e.g. `use std::prelude::rust_2021::*;`).][86294]

Compiler
--------
- [Added tier 3\* support for `powerpc64le-unknown-freebsd`.][83572]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Updated std's float parsing to use the Eisel-Lemire algorithm.][86761]
  These improvements should in general provide faster string parsing of floats,
  no longer reject certain valid floating point values, and reduce
  the produced code size for non-stripped artifacts.
- [`string::Drain` now implements `AsRef<str>` and `AsRef<[u8]>`.][86858]

Stabilised APIs
---------------

- [`Bound::cloned`]
- [`Drain::as_str`]
- [`IntoInnerError::into_error`]
- [`IntoInnerError::into_parts`]
- [`MaybeUninit::assume_init_mut`]
- [`MaybeUninit::assume_init_ref`]
- [`MaybeUninit::write`]
- [`array::map`]
- [`ops::ControlFlow`]
- [`x86::_bittest`]
- [`x86::_bittestandcomplement`]
- [`x86::_bittestandreset`]
- [`x86::_bittestandset`]
- [`x86_64::_bittest64`]
- [`x86_64::_bittestandcomplement64`]
- [`x86_64::_bittestandreset64`]
- [`x86_64::_bittestandset64`]

The following previously stable functions are now `const`.

- [`str::from_utf8_unchecked`]


Cargo
-----
- [Cargo will now deduplicate compiler diagnostics to the terminal when invoking
  rustc in parallel such as when using `cargo test`.][cargo/9675]
- [The package definition in `cargo metadata` now includes the `"default_run"`
  field from the manifest.][cargo/9550]
- [Added `cargo d` as an alias for `cargo doc`.][cargo/9680]
- [Added `{lib}` as formatting option for `cargo tree` to print the `"lib_name"`
  of packages.][cargo/9663]

Rustdoc
-------
- [Added "Go to item on exact match" search option.][85876]
- [The "Implementors" section on traits no longer shows redundant
  method definitions.][85970]
- [Trait implementations are toggled open by default.][86260] This should
  make the implementations more searchable by tools like `CTRL+F` in
  your browser.
- [Intra-doc links should now correctly resolve associated items (e.g. methods)
  through type aliases.][86334]
- [Traits which are marked with `#[doc(hidden)]` will no longer appear in the
  "Trait Implementations" section.][86513]


Compatibility Notes
-------------------
- [std functions that return an `io::Error` will no longer use the
  `ErrorKind::Other` variant.][85746] This is to better reflect that these
  kinds of errors could be categorised [into newer more specific `ErrorKind`
  variants][79965], and that they do not represent a user error.
- [Using environment variable names with `process::Command` on Windows now
  behaves as expected.][85270] Previously using envionment variables with
  `Command` would cause them to be ASCII-uppercased.
- [Rustdoc will now warn on using rustdoc lints that aren't prefixed
  with `rustdoc::`][86849]

[86849]: rust-lang/rust#86849
[86513]: rust-lang/rust#86513
[86334]: rust-lang/rust#86334
[86260]: rust-lang/rust#86260
[85970]: rust-lang/rust#85970
[85876]: rust-lang/rust#85876
[83572]: rust-lang/rust#83572
[86294]: rust-lang/rust#86294
[86858]: rust-lang/rust#86858
[86761]: rust-lang/rust#86761
[85769]: rust-lang/rust#85769
[85746]: rust-lang/rust#85746
[85305]: rust-lang/rust#85305
[85270]: rust-lang/rust#85270
[84111]: rust-lang/rust#84111
[83918]: rust-lang/rust#83918
[79965]: rust-lang/rust#79965
[87370]: rust-lang/rust#87370
[87298]: rust-lang/rust#87298
[cargo/9663]: rust-lang/cargo#9663
[cargo/9675]: rust-lang/cargo#9675
[cargo/9550]: rust-lang/cargo#9550
[cargo/9680]: rust-lang/cargo#9680
[cargo/9663]: rust-lang/cargo#9663
[`array::map`]: https://doc.rust-lang.org/stable/std/primitive.array.html#method.map
[`Bound::cloned`]: https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.cloned
[`Drain::as_str`]: https://doc.rust-lang.org/stable/std/string/struct.Drain.html#method.as_str
[`IntoInnerError::into_error`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_error
[`IntoInnerError::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_parts
[`MaybeUninit::assume_init_mut`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_mut
[`MaybeUninit::assume_init_ref`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_ref
[`MaybeUninit::write`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.write
[`Seek::rewind`]: https://doc.rust-lang.org/stable/std/io/trait.Seek.html#method.rewind
[`ops::ControlFlow`]: https://doc.rust-lang.org/stable/std/ops/enum.ControlFlow.html
[`str::from_utf8_unchecked`]: https://doc.rust-lang.org/stable/std/str/fn.from_utf8_unchecked.html
[`x86::_bittest`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittest.html
[`x86::_bittestandcomplement`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandcomplement.html
[`x86::_bittestandreset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandreset.html
[`x86::_bittestandset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandset.html
[`x86_64::_bittest64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittest64.html
[`x86_64::_bittestandcomplement64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandcomplement64.html
[`x86_64::_bittestandreset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandreset64.html
[`x86_64::_bittestandset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandset64.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
relnotes Marks issues that should be documented in the release notes of the next release. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
10 participants